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2008, observed the grant 
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reviewers, and agency officials. 
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HHS should take steps to ensure a 
fair and transparent award process, 
including clarifying evaluation 
criteria; selecting better qualified 
reviewers; better documenting 
agency guidance to reviewers and its 
rationale for grant decisions; and 
providing this rationale to applicants 
in a timely manner. 

HHS agreed to improve its process 
in response to all of GAO’s 
recommendations except the ones 
related to clarifying evaluation 
criteria and better documenting 
agency guidance to reviewers. 



View GAO-10-335 or key components. 
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(202) 512-7215 or brownke@gao.gov. 



Based on GAO’s review of past grant announcements for these programs, GAO 
found that the criteria upon which grant applications were evaluated were not 
clearly identified or presented in a single location in the announcement. 
Rather, GAO found that criteria were scattered throughout various sections of 
the announcement, had multiple labels, and were not presented in an orderly 
manner. As a result, applications that did not address the criteria from all 
sections were likely to receive lower evaluation scores, decreasing their 
chances of receiving a grant. 

HHS provides technical assistance to potential applicants for runaway and 
homeless youth grants, as required by statute. Of the 20 applicants GAO 
interviewed who sought technical assistance, 17 were satisfied with the help 
they received. 

Grant award decisions are primarily based on the results of the peer review 
process, and internal controls in place to ensure that applications are 
evaluated consistently were not always adequate. GAO found weaknesses in 
four out of the six procedures the agency relies on to ensure consistent 
evaluation of applications. For example, although HHS policy requires peer 
reviewers to be experts in the field of runaway and homeless youth programs, 
about one- quarter of the reviewers who evaluated applications for 2009 Street 
Outreach grants had little or no experience in this area. 

With regard to notification of grant award decisions, GAO found that they 
have not always been communicated to applicants in a timely manner, which 
can delay the start of new programs and present planning challenges for 
existing ones. GAO also found that the information in notification letters to 
applicants who were not awarded grants was not always clear or complete. 



HHS’s Grant Programs for Runaway and Homeless Youth 




BASIC CENTER 



Provides food, clothing, shelter, 
health care, and counseling 
services for runaway and 
homeless youth under age 18. 




TRANSITIONAL LIVING/ 
MATERNITY GROUP HOMES 



Provides runaway and homeless 
youth ages 16 through 21 with 
up to 18 months of residential 
services. 




STREET OUTREACH 



Designed to prevent the sexual 
abuse and exploitation of 
runaway and homeless youth 
and to treat any already 
abused or exploited. 



Source: GAO analysis of ACF documents. 
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May 10, 2010 

The Honorable Patrick J. Leahy 
Chairman 

The Honorable Jeff Sessions 
Ranking Member 
Committee on the Judiciary 
United States Senate 

The Honorable George Miller 
Chairman 

The Honorable John P. Kl ine 
Ranking Member 

Committee on Education and Labor 
House of Representatives 

According to the Department of Health and Human Services (HHS), 
thousands of U.S. youth run away from home, are asked to leave their 
homes, or become homeless each year. 1 Without shelter and guidance, 
these youth are vulnerable to exploitation and involvement in illicit 
activities, such as selling drugs and prostitution. In 2008 and 2009, HHS 
awarded more than $100 million each year in discretionary grants to 
providers of shelter and services for this vulnerable population through 
three programs for runaway and homeless youth. 2 The Basic Center and 
Transitional Living Programs fund short-term and longer-term shelter for 
youth, respectively, and the Street Outreach Program funds services to 
prevent sexual abuse and exploitation of these youth. 

Because only about one out of three homeless and runaway youth grant 
applications is approved, it is particularly important that the agency have 
systems in place to help ensure consistent and transparent grant award 
decisions. The Reconnecting Homeless Youth Act of 2008 mandated a 



'The exact number of children and youth that run away or become homeless is unknown 
due to the transient nature of this population and the lack of a consistent definition of a 
“runaway or homeless” individual. See CRS Report for Congress, Runaway and Homeless 
Youth: Demographics and Programs, RL33785 (Washington, D.C.: Mar. 19, 2009). 

According to the HHS Grants Policy Statement, discretionary grants are those that permit 
the federal government, according to specific authorizing legislation, to exercise judgment 
in selecting the recipients. Discretionary grants are generally made following a competitive 
process. 
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GAO review of HHS’s process for awarding certain Runaway and 
Homeless Youth Act grants. 3 This report responds to that mandate by 
addressing the following questions: (1) How clear are grant 
announcements and application documents and requirements? (2) How 
useful do applicants find the technical assistance they receive to assist 
them with applying for grants? (3) How are grant award decisions made 
and to what extent does this process comply with federal requirements? 
(4) To what extent are grant award decisions communicated to applicants 
in a timely and clear manner in accordance with federal requirements? 

To answer these questions, we reviewed relevant laws, regulations, grant 
announcements, applications and other agency documents, and we 
interviewed key agency officials and contractors. To determine applicants’ 
views on the usefulness of technical assistance, we randomly selected and 
interviewed 24 out of the 590 applicants that competed for fiscal year 2008 
grants. This sample included applicants who were awarded grants and 
applicants who were not. We also randomly selected and interviewed 6 
peer reviewers out of approximately 170 who evaluated applications for at 
least one of the three grant programs for fiscal year 2008. Additionally, we 
interviewed representatives from the National Network for Youth, an 
organization that represents providers of services to youth and families. 

To determine how grant award decisions are made, we analyzed agency 
data and documents related to peer review evaluations and final award 
decisions for the three programs for fiscal years 2007 and 2008. We also 
observed the peer review process for the Street Outreach Program in June 
2009. 4 To determine the extent to which the grant award process complies 
with federal requirements, we compared the grant award process to 
relevant requirements in the law, HHS regulations and written guidance, 
and internal control standards for the federal government. 5 To determine 
the extent to which grant award decisions are communicated in a timely 
and clear manner in accordance with federal requirements, we reviewed a 
random sample of 69 notifications that were sent to applicants for at least 
one of the three programs for fiscal year 2008 grants. Additionally, we 
established a timeline for each grant program’s award process for fiscal 
years 2007 and 2008. 



3 Pub. L. No. 110-378, § 9. 

Approximately 180 applications were reviewed by 19 peer review panels during that 
competition. 

5 Standards for Internal Control in the Federal Government, GAO/AIMD-OO-21.3.1 
(Washington, D.C.: November 1999). 
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We conducted this performance audit from March 2009 through April 2010 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 



Background 



Program Descriptions The Runaway and Homeless Youth Act 6 , as amended, authorizes federal 

funding in the form of discretionary grants for three programs to assist 
runaway and homeless youth. These programs are administered by the 
Family and Youth Services Bureau (FYSB) within HHS’s Administration 
for Children and Families (ACF). 7 The three programs — the Basic Center 
Program, Transitional Living Program, and Street Outreach Program 8 — 
enable local public and private organizations and shelters in all 50 states 
and the U.S. territories to compete for grants that allow them to serve 
runaway, homeless, and sexually exploited youth who may be on the 
streets and in need of shelter or longer-term support. 

The Basic Center Program provides temporary shelter, counseling, and 
other services to runaway and homeless youth under the age of 18. Basic 
Center grants are awarded competitively to providers and may be awarded 
for a period of up to 3 years. 9 The Transitional Living Program provides 
homeless youth ages 16 through 21 with longer-term residential services 
for up to 18 months. 10 These services include such things as counseling 



6 The Runaway and Homeless Youth Act was enacted as Title III of the Juvenile Justice and 
Delinquency Prevention Act of 1974 (Pub. L. No. 93-415). 

The Administration for Children and Families (ACF) is 1 of 11 federal agencies within 
HHS. 

s The Street Outreach Program is formally known as the Education and Prevention Services 
to Reduce Sexual Abuse of Runaway, Homeless, and Street Youth Program. 

9 Basic Center funds are allotted to states on the basis of their relative population of youth 
less than 18 years of age. The term “state” includes the District of Columbia and Puerto 
Rico. U. S. territories also receive funding based on their population of youth. 

^Transitional Living Program also includes grants for maternity group homes targeted to 
young mothers and their children to meet the needs of this population. 
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and education in basic life skills, interpersonal skills, educational 
advancement, job attainment skills, and physical and mental health care. 
Transitional Living grants are awarded competitively to providers and may 
be awarded for up to 5 years. The Street Outreach Program provides 
education, treatment, counseling, and referrals for runaway, homeless 
youth under the age of 18 who have been subjected to or are at risk of 
being sexually abused and exploited. Street Outreach grants may be 
awarded for up to 3 years. See figure 1 for key aspects of these three 
programs. 



Figure 1: HHS’s Grant Programs for Runaway and Homeless Youth 
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Source: GAO analysis of ACF documents. 
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For fiscal years 2002 through 2009, funding for these programs has been 
just over $100 million in total, with Basic Center funding representing the 
largest dollar amount authorized of the three grant programs. Funding for 
these programs over the past several years is shown in table 1. 



Table 1 : Funding for Runaway and Homeless Youth Program, Fiscal Years 2002 to 2008 



(Dollars in thousands) 



Program 


2001 


2002 


2003 


2004 


2005 


2006 a 


2007 a 


2008 b 


Basic Center 


$48,338 


$48,288 


$48,298 


$49,171 


$48,786 


$48,265 


$48,298 


$52,860 


Transitional Living 


20,740 


39,736 


40,505 


40,260 d 


39,938° 


39,51 1 d 


39,539°’ d 


43,268 d 


Street Outreach 


14,999 


14,999 


15,399 


15,302 


15,178 


15,017 


15,027 


17,221 


Total 


$84,127 


$103,023 


$104,202 


$104,733 


$103,902 


$102,793 


$102,864 


$113,349 



Source: CRS Report to the Congress, Runaway and Homeless Youth: Demographics and Programs, RL33785, March 19, 2009 
(Washington, D.C.). 

Note: Basic Center Program and Transitional Living Program funding are distributed under the 
Consolidated Runaway and Homeless Youth Program. Street Outreach Program funds are 
distributed separately. 

“The fourth Continuing Resolution for the FY2007 budget (Pub. L. No. 1 1 0-5) generally funded 
programs at their FY2006 levels. However, the FY2006 funding total for the Runaway and Homeless 
Youth Program was slightly lower than the FY2007 total because of an additional transfer of funds 
from the Runaway and Homeless Youth Program accounts to an HHS sub-agency. 

“The FY2008 appropriations include a 1.7 percent across-the-board rescission on Labor-HHS- 
Education programs. 

“The Department of Health and Human Services, in consultation with Congress, will allocate the $97.2 
million for the Basic Center Program and Transitional Living Program between the two programs. 

“ Includes funding for the Maternity Group Home component. 



Grant Award Process HHS’s grant award process for Runaway and Homeless Youth grants is 

comprised of several major steps. HHS’s Family and Youth Services 
Bureau, which is referred to as the Program Office, and the Grants 
Management Office are responsible for carrying out and overseeing this 
process. Some of the steps in the grant award process are performed by 
contractors on behalf of the agency, and one step in the process is 
performed by panels of peer reviewers selected by the agency to evaluate 
grant applications. 

The grant award process consists of the major steps as illustrated in figure 2. 

• Grant Announcement: Each fiscal year, the agency develops and 

publishes a grant announcement for each grant program. Announcements 
provide the information potential applicants need to determine if they are 
eligible to apply and the instructions on how to complete and submit their 
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application. In addition, they include the criteria used to evaluate 
applications. 

• Technical Assistance: Each announcement lists the technical service 
providers responsible for providing technical assistance to potential 
applicants to help them understand the announcement requirements. 
Technical assistance can be provided through a webinar, seminar, 
information on the Web, or upon request. 

• Application Submission: Applications may be submitted electronically 
via Grants.gov or by hard copy via mail or other delivery service. 
Applicants may also hand deliver their application to the agency’s 
contractor responsible for receiving the applications. The deadline for 
submitting applications is usually 45 to 60 days after an announcement is 
published. 

• Application Pre-Screening: Applications are prescreened to determine 
whether they meet two requirements. Applications are eliminated from 
review if they are received after the deadline or if they request more 
funding than the maximum amount specified in the announcement. 

• Peer Review of Applications: Applications remaining after pre- 
screening are submitted to peer review panels. A panel generally consists 
of three peer reviewers, who apply the evaluation criteria contained in the 
announcement to applications and score the applications, and a panel 
chair responsible for facilitating consensus of the peer review panel. 



Peer reviewers assign points to each application, based upon specific 
criteria that are outlined in the announcement. The points are added up 
and the applicant’s average score is derived. This score, ranging from 0 to 
100, becomes the basis for the ranked listing of applicants which the 
agency uses in its award decisions. Because applicants whose score places 
them below the total available funding limit may be denied a grant, a single 
point can make a difference between awarding a grant and denying a 
grant. 

• Final Grant Award Decisions: Taking into account peer review panel 
scores and comments for each application and, in some cases, other 
factors, the Program Office and the Grants Management Office make the 
final award decisions. These decisions are documented in the final funding 
award decision memos, which contain the listing of all applicants, ranked 
by their scores, and the final award decisions. 
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• Notification of Award Decisions: Each applicant is sent a letter that 
communicates the grant award decision. Successful applicants are notified 
before letters are sent to unsuccessful applicants. 



Figure 2: HHS’s Runaway and Homeless Youth Grant Award Process 




Source: GAO analysis of ACF documents. 
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Evaluation of Applications According to the HHS Grants Policy Statement, each discretionary grant 

application, including those for Runaway and Homeless Youth Grants, 
must receive an objective, . .advisory review.. .by a minimum of three 
unbiased reviewers with expertise in the programmatic area for which 
applications are submitted.” To meet this requirement, the agency relies on 
the peer review process, in which three reviewers convene to evaluate and 
score applications based on the criteria outlined in the announcement. The 
peer reviewers are defined by the agency as experts in the field of runaway 
and homeless youth programs. Figure 3 provides an overview of the peer 
review process. 



Figure 3: Runaway and Homeless Youth Grants Peer Review Process 
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Source: GAO analysis of ACF documents. 



HHS awarded grants to about one-quarter of the applicants that applied in 
2007 and 2008, as shown in table 2. 
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Table 2: Numbers of Applications for 2007 and 2008 Runaway and Homeless Youth Grants and Dispositions 



Applications received Applications funded Applications not funded 



Program 


FY 2007 


FY 2008 


FY 2007 


FY 2008 


FY 2007 


FY 2008 


Basic Center 


230 


209 


130 


120 


100 


89 


Transitional Living 


281 


338 


72 


85 


209 


253 


Street Outreach 


186 


144 


86 


21 


100 


123 


Total 


697 


691 


288 


226 


409 


465 



Source: GAO analysis of ACF documents. 



Grant 

Announcements Have 
Not Always Provided 
Clear and Concise 
Information 



Based on the grant announcements we reviewed and our observation of 
the peer review process, the criteria upon which grant applications would 
be evaluated were not clearly defined in a single location in the 
announcement. Rather, we found that criteria were scattered throughout 
various sections of the announcement, had multiple labels, and were not 
presented in an orderly manner in a single location. For example, for the 
2009 Street Outreach Program grant competition, grant applications were 
evaluated and scored based on how well they addressed criteria contained 
in three different sections of the announcement. First, applicants must 
address the “Program Requirements,” found in Section 1 of the 
announcement. Second, applicants must address the “Project 
Description,” found in Section 4. Third, applicants must address the 
“Evaluation Criteria” found in Section 5. However, only the “Evaluation 
Criteria” section of the announcement explicitly described how their 
responses would be evaluated and scored. Because the applicant must 
address criteria contained in different sections of the announcement, if 
applicants focused primarily on responding to the “Evaluation Criteria” 
they may not have adequately addressed information in the other sections. 
If the applicants focused only on the “Evaluation Criteria,” these 
applicants could have received lower scores, which would have decreased 
their likelihood of being awarded grants. Figure 4 represents the various 
locations where we found descriptions of criteria. 
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Figure 4: Depiction of Scattered Criteria in Different Sections of the Grant Announcement 
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Source: GAO analysis of ACF documents. 



Additionally, peer reviewers we interviewed noted that consolidation of 
criteria in the announcement into a single location would aid them in their 
evaluation of applications by reducing the time it would take to review the 
application because they would not need to look in multiple places in the 
application for information. 11 

During our observation of the 2009 Street Outreach Program grant 
competition, we found that the agency provided detailed guidance to peer 
reviewers to help them evaluate and score applications. This guidance, 
which was not available to applicants, consolidated information from 
various sections of the announcement. The federal officials instructed 
reviewers to focus on specific information when evaluating and scoring 
applications. Because applicants did not have this detailed guidance, 
which combined information from various parts of the announcement, 
applicants may not have had full knowledge of what information was 



^Another HHS agency, The National Institutes of Health (NIH) conducted a peer review 
self-study in 2007 to improve the quality and transparency of its grants review process for 
research grants and cooperative agreements. As a result of the study, NIH shortened the 
length of its applications and aligned it with specific review criteria to clearly identify for 
applicants the most important requirements to address and reduce the burden of review for 
reviewers. 
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critical to receiving a high score. Table 3 shows examples of the guidance 
provided to peer reviewers during the 2009 Street Outreach Program grant 
competition. 



Table 3: Examples of Guidance Provided to Peer Reviewers for Evaluation of Applications 



Criteria Guidelines 

(from announcement) (from other parts of the announcement) 

Describes a clear need for the proposed project through a Does the application describe the conditions of youth and families 

discussion of the conditions of youth and families in the area to be in the area to be served that would identify and support the need 

served. for the project? 

Does the application describe need for assistance by identifying 
the physical, economic, social, financial, institutional, and/or other 
problems requiring a solution? 

Does the emergency preparedness plan include prevention, 
preparedness, response, and recovery efforts/activities? 

Does the plan include strategies for addressing evacuation, 
security, food, medical supplies and notification of youth families? 
What is the identified alternative location to deliver services in the 
case of fire or loss of use of the facility? 

What is the plan to notify FYSB of evacuation plans when they are 
executed? 



Describes emergency preparedness and management plan by 
addressing steps to be taken in care of a local or national 
situation that poses risk to the health and safety of program staff 
and youth. 



Source: GAO analysis of 2009 Street Outreach Program grant competition documents. 



ACF Provides 
Technical Assistance 
That Applicants 
Found Helpful 



ACF provides technical assistance to potential applicants for runaway and 
homeless youth grants, as required by statute. 12 Technical assistance is 
generally defined as providing expertise or support to applicants and 
grantees for the purpose of strengthening their capabilities for providing 
shelter and support services for runaway and homeless youth. In fiscal 
years 2006 and 2007, the agency provided technical assistance to potential 
applicants through its regional network of 10 providers, and listed these 
providers in its announcements. 



However, beginning in September 2007, ACF centralized its technical 
assistance in order to provide more consistent technical assistance for all 
applicants, regardless of where they were located. At that time, the agency 



12 The statutory requirement calls for HHS to provide “informational assistance to potential 
grantees.” 42 U.S.C. § 5714a HHS terms the assistance it provides as “technical assistance.” 
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entered into cooperative agreements with the University of Oklahoma to 
provide technical assistance nationwide. 13 

Through its providers, the agency coordinates technical assistance, which 
generally consists of a pre-application conference (webinar) covering the 
application requirements such as the project description, eligibility, and 
the evaluation criteria, among other things. After the conference, a 
recording and transcript is posted on the agency Web site. 

Potential applicants may also ask specific questions of the contacts listed 
on the announcement. These contacts include the Program Office officials 
and technical assistance providers. If the technical assistance providers 
cannot answer the questions, they coordinate with agency staff to obtain 
responses that are then posted to the Web site. The technical assistance 
providers also arrange seminars on broader topics related to runaway 
youth, such as mental health, crisis intervention, and skills training. 

Most of the applicants we interviewed who received technical assistance 
under both systems reported that they found it helpful. For example, 17 of 
the 20 applicants who sought technical assistance were satisfied with the 
help they received. However, three of these applicants said they prefer the 
technical assistance provided by their regional providers because of such 
things as the regional assistance being more “hands-on,” the regional staff 
being more responsive and accessible, and the regional staff being more 
knowledgeable of local programs. Agency officials noted that the agency 
has moved toward centralized approach to gain a more consistent 
approach to the technical assistance it provides. 



13 The University of Oklahoma established and operates the Runaway and Homeless Youth 
Training and Technical Assistance Center (RHYTTAC) to provide these services. 
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ACF’s Process for 
Determining Grant 
Awards May Not 
Ensure Consistent 
Decisions 



ACF’s Peer Review 
Process Has Weak Internal 
Controls 



ACF’s process for determining which grant applicants will be awarded 
grants is primarily based on the results of the peer review process, which 
has weak internal controls to ensure that applications are evaluated 
consistently. According to GAO standards, internal controls should 
provide reasonable assurance that the agency’s objectives, such as 
providing grants to the most qualified providers, are being achieved. 14 
Ideally, internal controls should be continuous, built-in components of the 
agency’s processes, and should provide reasonable assurance that the 
grant award process works as it is designed to work. Our review of ACF’s 
grant award process found that, while the agency has a number of internal 
controls in place to help ensure consistent application of evaluation 
criteria across reviewers and across panels, some of these controls are 
limited in their effectiveness. For example, we found weaknesses in four 
out of six internal controls related to the grant award process, as shown in 
table 4. 



14 Standards for Internal Control in the Federal Government, GAO/AIMD-OO-21.3.1 
(Washington, D.C.: November 1999). 
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Table 4: Examples of Internal Controls and Related Weaknesses 





Internal control 


Weakness 


1 . 


Peer reviewers responsible for evaluating grant applications 
should be experts in the field of runaway and homeless youth 
programs 


Some peer reviewers had little or no related expertise in the 
field. 


2. 


The agency holds an orientation session for all reviewers and 
panel chairs on the first day of panels. Additionally, it holds a 
meeting for panel chairs on how to apply the evaluation criteria. 
The agency also holds a session for new reviewers. 


Not all peer reviewers attended these meetings. 


3. 


Detailed guidance is provided to peer reviewers to aid them in the 
evaluation of applications. 


Not all reviewers followed this guidance because it was not 
always found in the criteria section of the announcement. 
This could lead to inconsistency in scoring applications. 


4. 


Federal officials review each panel’s scores and narrative 
comments and provide each panel with feedback to help improve 
the application of criteria. 


This feedback is not documented in a permanent record. 



Source: GAO analysis of ACF documents and observation of peer review process. 



First, ACF does not always select peer reviewers whose qualifications 
comply with the standards outlined in HHS policy. The policy states that 
each application for runaway and homeless youth grants must receive an 
objective, advisory review by a minimum of three unbiased reviewers with 
expertise in the programmatic area for which applications are submitted. 
Furthermore, the announcements we reviewed stated that grant 
application reviewers should be experts in the field of runaway and 
homeless youth programs. However, we found that HHS considered 
students, school teachers, business consultants, and television and media 
workers as qualified peer reviewers. Our review of resumes of all the peer 
reviewers and chairs for 2009 Street Outreach Program grants found that 
many had professional and volunteer experiences that were not always 
directly related to runaway and homeless youth programs. Based on the 
resumes of 76 peer reviewers, we found that 26 peer reviewers had direct 
experience with runaway and homeless youth programs listed on their 
resume, and another 31 had indirect experience, such as social work, 
teaching, or grant reviewing. However, 19 did not appear to have any of 
the relevant knowledge and expertise in runaway and homeless youth 
programs required by HHS policy. Three of these 19 reviewers were 
identified as “youth reviewers” in their resumes. One agency official 
responsible for the grant review process during 2009 explained that HHS 
interprets its policies governing peer reviewer qualifications broadly and 
accepts all related experience. He also noted that HHS encourages the use 
of “youth reviewers” for its peer review panels. 
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Second, during our observation of the 2009 Street Outreach Program grant 
competition, we found that the meetings for peer reviewers and chairs 
were not mandatory. These meetings included an orientation session, 
panel chair meetings, and new reviewer meetings. Meetings for panel 
chairs are particularly important for helping to ensure consistent 
evaluations across panels because in these meetings, all panel chairs agree 
on how to apply the evaluation criteria. However, we observed that some 
panel chairs did not attend these meetings, and, therefore, their panels 
may not have applied the evaluation criteria in the same manner as panels 
whose chairs had attended the meetings. Similarly, new reviewers were 
permitted to miss the new reviewers’ meetings and still participate in the 
reviews, which could also increase the risk of inconsistent application of 
evaluation criteria. 

Third, we observed that the agency provided detailed guidance to peer 
reviewers to aid them in evaluating applications. The detailed guidance 
provided to reviewers explaining the evaluation criteria has led to 
variation in application of criteria by review panels. For example, when we 
observed peer review panel deliberations for the 2009 Street Outreach 
grants, we found that peer review panels varied in the way they applied 
the criterion for evaluation of emergency evacuation plans. The 
announcement’s “Evaluation Criteria” section contained the following 
evaluation criterion related to emergency plans: 

The application “describes the emergency preparedness and management 
plan by addressing steps to be taken in case of a local or national situation 
that poses risk to the health and safety of program staff and youth.” 

At the panel session, agency officials told panels that they should also 
apply all of the information in the detailed guidance they were given, 
which included information in the “Program Requirements” section of the 
announcement. Federal officials advised peer reviewers that they should 
score the application on the following information: 

“Grantees must immediately provide notification to FYSB when 
evacuation plans are executed.” 

As a result, peer review panels that followed the guidance gave lower 
scores to applicants that did not specifically indicate that they would 
notify the agency when an evacuation occurred. One peer review panel we 
observed, however, did not give lower scores when this was not specified 
in an application. These peer reviewers said that they did not think it was 
fair to assign lower scores in these cases because the more detailed 



Page 15 



GAO-10-335 Runaway and Homeless Youth Grants 




information about evacuation requirements was not listed in the 
“Evaluation Criteria” section of the announcement. 

Additionally, we interviewed peer reviewers who participated in panels for 
2008 runaway and homeless youth grants. Three of the six peer reviewers 
we interviewed told us they observed variations in the way panels applied 
the criteria. Reviewers said that the 2008 Transitional Living Program 
announcements contained evaluation criteria requiring applicants to 
provide background checks for all staff members who would be working 
with youth. However, the peer reviewers told us that the guidance 
provided to peer reviewers by the agency during that review process 
further specified that these background checks must be conducted in 
accordance with local, state, and national requirements. According to the 
peer reviewers we interviewed, this could have led to variation in how this 
aspect of the application was evaluated by different panels. Given that the 
peer review score is the key factor in determining grant awards, 
inconsistent evaluation criteria across panels can have a significant impact 
on whether an applicant is awarded a grant or not. 

The fourth control weakness we observed during our review of the 2009 
Street Outreach Program grant competition was that agency officials did 
not keep a permanent record of their comments and feedback to peer 
review panels during their oversight of the peer review process, which 
introduced further potential for inconsistent application of evaluation 
criteria. Agency officials review the panel’s scores and narrative comments 
for each application during the peer review process before they are 
finalized. The officials visit panels as peer reviewers deliberate and 
respond to their questions, and provide feedback to chairpersons on their 
panel’s evaluations. Agency officials told us their review and feedback is 
meant to ensure that all panels apply the evaluation criteria in the same 
way. 16 However, the federal officials we observed did not record this 
information in a permanent record. Instead, the officials provided their 
feedback to the chair via comments written on post-it notes. This lack of 
permanent documentation of federal official feedback to peer review 
panels makes it difficult for the agency to ensure that it is providing 
consistent guidance to panels and responding to problems across panels in 
the same way during the peer review process. 



15 They also noted that this is a method for the agency to determine how individual 
reviewers are performing and if they should be selected for future reviews. 
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No weaknesses were apparent in two of the six internal controls-(l) the 
provision of standard training materials to peer reviewers prior to panel 
sessions, and (2) the presence of federal officials on site during panel 
sessions to respond to questions from, and communicate on a daily basis 
with panels. 



The Basis for Denying Final funding decision memos used to internally document grant award 

Grants Is Not Always decisions for 2007 and 2008 did not contain supporting information 

Documented regarding why applications with high scores were not funded. Final 

decisions regarding grant awards are determined by HHS’s Program Office 
and Grants Management Office, taking into account the review panels’ 
scores and narrative comments for each application. According to HHS 
policy and guidance, the agency has the discretion to deny a grant to an 
applicant who would otherwise receive one based on the results of the 
peer review score alone. The agency is permitted to use its discretion to 
deny grants based on other reasons, such as the agency’s concerns about 
the applicant’s program or about the concentration of service providers in 
the applicant’s location, which is referred to as concerns about 
“geographic distribution” of services. However, the agency does not 
always clearly document the rationale for its decision to deny a grant 
based on “geographic distribution” of services. 16 When grants were denied 
for geographic reasons in 2007 and 2008, we found that the final funding 
decision memos did not clearly describe the details surrounding such 
denials, such as the number of other programs that exist in the same 
locale, the services they provide, or the numbers of youth they serve. Such 
details could support or justify a denial for geographic reasons. Without 
fully documenting and permanently recording its rationale for exercising 
its discretion to deny grants to highly scored applicants, the agency 
decision-making process is not transparent. 



16 The agency uses this discretion to ensure that services are geographically distributed 
throughout the nation and to increase the capacity of services to communities with a high 
concentration of runaway and homeless youth. 
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Notification of Grant 
Award Decisions 
Have Not Always 
Been Timely, and 
Notices Are Not 
Always Clear 



Notification Time Frames 
Can Present Planning 
Challenges for Some 
Applicants 



Grant award decisions are not always communicated in a timely manner, 
which may present planning challenges for some applicants. According to 
one ACF official, successful applicants are generally notified at the end of 
the federal fiscal year. Based on our review of grant documents for fiscal 
years 2007 and 2008, we found that for all but the 2008 Transitional Living 
Program grants, this was true, regardless of when the announcement 
closed or when the funding decisions were made. For example, 
applications for the 2008 Basic Center Program were due in February 2008 
and were evaluated and scored in March; however, applicants were not 
notified of their award status until September, 6 months later. HHS policy 
does not indicate when notification letters should be distributed to 
applicants, but according to an ACF official, awards to successful 
applicants are made by September 30 because most new programs are 
expected to start on or before October 1. Given the proximity of the 
notification date to program start date, some successful applicants with 
new programs we spoke with told us that the September notification 
timeframe did not allow enough preparation time to hire staff and secure 
the resources needed to provide services. See figure 5 for the timeline of 
dates for key events for the fiscal year 2007 and 2008 grant award process. 
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Figure 5: Timelines of Key Dates for the Grant Award Process, Fiscal Years 2007 and 2008 
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• Decision September 3 

• Notification 
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Source: GAO analysis of ACF documents. 



“The agency shifted its funding schedule for the Transitional Living Program in 2008. 
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Notification delays also create planning issues for ongoing programs that 
are not awarded new grants and, as a result, need to develop contingency 
plans for continuing or discontinuing services. Since unsuccessful 
applicants are notified of their grant award status after successful 
applicants have been notified, an applicant whose previous grant is about 
to expire may experience planning problems if notifications are delayed. 
Delays in notifying unsuccessful applicants may not give applicants 
adequate time to react to not being awarded a new grant. In the event that 
funding is denied or discontinued, earlier notification of award decisions 
could help providers properly plan. 

According to an ACF official, there is nothing in policy that prohibits 
notifying an applicant as soon as award decisions have been made. The 
official told us that delays in sending out notification letters are linked to 
the timeliness of writing and issuing the announcement. According to this 
official, announcements must be reviewed by many departments within 
the agency, and, therefore, the turnaround time is not as timely as it could 
be. However we found that even after the announcements were published 
and closed, applicants were still not notified of their award status for 
several months. For example, for the 2008 Transitional Living grant, 
regardless of when the announcement was published, applicants were not 
notified of their award, until close to 4 months after the panels had 
completed evaluating the applications. Similarly, notifications of decisions 
related to 2008 Basic Center grants were not sent out until about 7 months 
after the panels. 

In addition to the challenges applicants experienced due to notification 
delays, the agency created additional planning challenges for applicants 
when it unexpectedly changed the timing of the funding cycle for the 
Transitional Living Program in fiscal year 2008 without notifying 
applicants of this change in a timely manner. The announcement stated 
that ACF anticipated making grant awards in the first quarter of fiscal year 
2008, which would have been from October through December of 2007. 
However, the grant award start date was changed to March 2008 after this 
announcement was published. According to an agency official, the original 
start date was moved in an effort to spread out the timing of peer review 
panels for each of the three runaway and homeless youth programs and 
other activities that were scheduled to occur around the same time during 
the summer months. As a result of moving the cycle start date — from 
October to March — some successful applicants were without federal 
funding for several months between the end of the previous grant cycle 
and the new grant award start date. 
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Runaway and homeless youth service providers have also raised concerns 
to their congressional representatives about the timeliness of notifications. 
Specifically, we reviewed nine complaint letters that were sent to 
congressional representatives regarding runaway and homeless youth 
grants applications in 2007 and 2008. One letter, representing six 
providers, stated that notification delays created planning problems for 
service providers who were not able to develop contingency planning for 
either the continuation or discontinuation of their programs. ACF 
responded to the complaint by noting that it offers funding for successful 
applicants to recoup some of the costs that programs incurred due to the 
delay. 

In addition, the National Network for Youth, an organization that 
represents providers of services to youth and families also noted that the 
timeliness of notifications has been an issue of concern for its 
membership. In particular, some service providers have raised issues 
about the difficulties receiving timely communications from ACF 
concerning grant awards. 



Notification Letters to 
Applicants Are Not Always 
Clear 



All of the successful applicants we spoke with felt that their notification 
letters were clear and contained sufficient information; however, 
unsuccessful applicants were not all satisfied with the clarity and 
completeness of the information presented in their letters. The standard 
letter to unsuccessful applicants may list several possible “other factors” 
for the denial, beyond their peer review panel score, without any 
indication of which of the reasons listed in the standard notice applied to 
their application. See appendix I for a standard letter. The “other factors” 
include: 



• “comments of reviewers and government officials,” 

• “staff evaluation and input,” 

• “geographic distribution,” and 

• “audit reports and previous program performance.” 

Some unsuccessful applicants told us the letter did not contain enough 
information for them to understand why their application was denied. In 
particular, some applicants told us that they did not understand what the 
agency meant by geographic distribution, which was the basis for denying 
grants to at least eight applicants during fiscal years 2007 and 2008. 
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Officials told us that “geographic distribution” means that an applicant was 
denied because the geographical area their program would serve is already 
served by another runaway and homeless youth service provider. The 
agency does not keep a record to document detailed information that 
would support or justify a denial for geographic reasons, such as the 
number or names of programs that exist in the same locale, the services 
these programs provide, or the numbers of youth they serve. As a result, it 
is not possible to verify that denying a grant based on “geographic 
distribution” was justified. 

Applicants who want further explanation of their award decisions may 
request additional information along with their scores from ACF through a 
Freedom of Information Act request. 17 An ACF official told us that it would 
be difficult to provide all unsuccessful applicants more information 
supporting the denial decision based on other factors such as “geographic 
distribution” in notification letters because of limited resources. However, 
the resources needed to provide such information may be small, given that 
“geographic distribution” was the basis for denying grants to only a small 
number of applicants during fiscal years 2007 and 2008. 

Moreover, based upon our review of decision notices sent to applicants 
who were screened out of the competition due to late submissions or 
improper funding requests, we found contradictory language that may 
confuse applicants. Specifically, the letter states that “the limited 
availability of funds permitted us to select only the highest scoring 
applications that also met all of the eligibility requirements,” leaving the 
impression that the application, in these cases, had been evaluated and 
scored by a peer review panel. However, applications that are screened- 
out of the process before the peer review session are not evaluated or 
scored. When we pointed out this statement to the agency, officials agreed 
the language could be confusing to applicants. 



Conclusions 



The runaway and homeless youth grant programs provide much needed 
services to a vulnerable population and the number of applications far 
exceed the number of grants that can be awarded with available funding. 



1 The Freedom of Information Act (5 U.S.C. § 552) generally provides that any person has a 
right, enforceable in court, to obtain access to federal agency records, except to the extent 
they are protected from public disclosure. 
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To ensure that ACF awards these grants to the most capable applicants, its 
award process must be fair and transparent. 

Without clearly organized evaluation criteria in grant announcements, 
applicants can have difficulty determining what their applications will be 
evaluated on. Furthermore, without consistent evaluation of applications 
in the process, there cannot be a level playing field for all applicants. All 
peer reviewers must have the required programmatic expertise, or not all 
applicants are evaluated by their peers. Additionally, unless all peer 
reviewers attend meetings at panel review sessions; these meetings cannot 
help ensure consistent evaluation of applications. Without documentation 
of ACF comments to peer review panels during the review process there is 
also a risk that the evaluation process will not be consistent. Moreover, 
without fully documenting the rationale for denying grants to highly 
scored applicants, agency grant award decisions are not transparent. 

Once the grant award decisions are made, it is incumbent on ACF to notify 
applicants of decisions in a timely manner and provide them with clear 
and specific information about, in particular, decision not to grant awards. 
Without such notification, applicants may experience planning challenges 
and not fully understand the reasons they were denied grants. 



Recommendations 



To enhance transparency and fairness in the grant award process, and 
improve grantees ability to plan for services, we recommend that the 
Secretary of Health and Human Services direct the Assistant Secretary for 
the Administration for Children and Families to take the following seven 
actions: 

Clearly identify in grant announcements all the criteria that peer reviewers 
will use to evaluate and score applications, and ensure that peer reviewers 
use only those criteria during the peer review process. 

Select peer reviewers with expertise in the programmatic area for which 
they are evaluating grant applications. 

Make all meetings for peer reviewers, including those for new reviewers 
and chairs, mandatory. 

Document and maintain records of ACF comments to peer review panels 
during the review process. 
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• Document the specific reasons for denying grants to high-scoring 
applicants in favor of other applicants for the agency record. 

• Provide clear information to applicants about the specific reasons their 
applications were not approved. 

• Notify applicants about the outcome of their applications as soon as grant 
award decisions are made. 



Agency Comments 



We provided a draft of this report to the Department of Health and Human 
Services for review and comment; these appear in appendix II. In its 
comments, HHS disagreed with our recommendation to review and revise 
announcements to ensure that all evaluation criteria listed be clearly 
labeled as evaluation criteria and be contained in a single section of the 
announcement. HHS maintains that all of the criteria used to evaluate and 
score applications are contained in section 5 of the announcement. 
However during the peer review process we observed, in addition to 
evaluating and scoring applications based on criteria specified in the 
“Evaluation Criteria” section of the announcement (section 5), some of the 
panels evaluated and scored applications based on criteria from two other 
sections of the announcement. Given the difference between the agency’s 
response to our recommendation and what we observed, we are revising 
our recommendation to highlight the need to ensure that the all criteria 
used to evaluate and score applications are clearly identified to applicants 
and peer reviewers, and that peer reviewers use only those criteria when 
evaluating and scoring applications. 

With regard to our recommendation to select peer reviewers with 
expertise in the program for which they are evaluating grant applications, 
HHS commented that the agency has elected to accept reviewers who are 
knowledgeable of the risk factors faced by runaway and homeless youth, 
and that many professional disciplines often intersect with runaway and 
homeless youth. However, we found that in the past the agency has used 
individuals that would not be expected to have relevant expertise, such as 
television and media workers. Noting our concern in this area, the agency 
indicated that they plan to take steps to ensure that all reviewers possess 
the knowledge and expertise in the particular program for which they are 
reviewing grant applications. In the event of a shortage of reviewers, the 
agency intends to staff panels with at least one peer reviewer with 
extensive relevant knowledge, which would continue to differ from the 
current policy that grants must receive an objective, advisory review by a 
minimum of “three” unbiased reviewers with expertise in the 
programmatic area for which applications are submitted. We agree that 
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professionals in varied disciplines could have sufficient expertise to serve 
as reviewers and recognize that it may be difficult for the agency to find 
enough reviewers with expertise in a particular program. As a result, we 
are clarifying our recommendation to include those that have expertise in 
the programmatic area for which they are evaluating grant applications, 
and not a specific program. 

Regarding our recommendations to make peer review meetings 
mandatory, HHS indicated that all meetings for peer reviewers and chairs 
are already mandatory but due to unforeseen factors, it is not always 
possible for all reviewers to attend. Indeed, during our observation of a 
peer review session, not all reviewers and chairs attended the meetings. 
Moreover, at the time, agency officials told us that attendance at these 
meetings was not explicitly mandatory, but highly encouraged. They also 
indicated that attendance was not enforced and attendance records were 
not maintained. In response to this recommendation, the agency indicated 
that they plan to officially notify all reviewers and chairpersons 
participating in future reviews that all training is mandatory. In the event 
some reviewers and chairpersons are not able to attend the mandatory 
training sessions due to unforeseen circumstances, the agency intends to 
offer “make up” sessions. 

HHS did not provide comments on our recommendation to maintain 
records of ACF comments to peer reviewer panels during the review 
process. However they agreed with our recommendation to document the 
specific reasons for denying grants to high-scoring applicants in favor of 
other applicants. HHS commented that the agency plans to include more 
details concerning geographic distribution in the letters to applicants who 
are denied grants for this reason. While these efforts would be in line with 
our recommendation; the details supporting such decisions must be 
consistently documented in the agency’s records to support the 
information provided to applicants in their letters. 

In response to our recommendation to provide clear information to 
applicants about specific reasons their applications were not approved, 
HHS stated that in accordance with ACF policy and procedures, every 
unsuccessful applicant is entitled to an explanation of why their 
application was not funded. In addition, the agency noted that, upon 
request, the Program Office will provide a debriefing to applicants. 
However, letters sent to unsuccessful applicants should clearly note that 
applicants may request a debriefing by the Program Office regarding 
specific reasons why their application was not funded. Currently, letters to 
unsuccessful applicants do not include this information. In addition, it is 
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important to revise the language in letters to applicants that are screened 
out of the grant competition that implies their application was evaluated 
and scored. 

Finally, HHS agrees with our recommendation to notify applicants about 
the outcome of their application as soon as grant award decisions are 
made. As part of the grant application process, the agency plans to 
explain to applicants that final grant decisions depend on the results of the 
grant award negotiations between ACF and the prospective grantees. We 
recognize that these grants are discretionary and that final award 
decisions involve negotiations that may take time. However, every effort 
should be made to complete negotiations and notify both successful and 
unsuccessful applicants as quickly as possible. To enable applicants to 
efficiently and effectively manage their programs, it is important for 
applicants to receive their notices in a timely manner. 

HHS also provided technical comments, which we incorporated into the 
report as appropriate. 



We are sending copies of this report to the Secretary of HHS, relevant 
congressional committees, and other interested parties. In addition, the 
report will be made available at no charge on GAO’s Web site at 
http://www.gao.gov. 

If you or your staffs have any questions about this report, please contact 
Kay E. Brown at (202) 512-7215 or brownke@gao.gov. GAO staff who 
made major contributions to this report are listed in appendix III. 




Kay E. Brown 
Director, Education, 

Workforce, and Income Security 
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Appendix I: Administration for Children and 
Families’ Notification Letters to Grant 
Applicants 



«Date» 

«AR Name» 

«AR Title» 

«Organization» 

«Address» 

«Address» 

RE: Grant Application: «Application Number». 

Dear «AR»: 

Thank you for submitting an application in response to the February 23, 2007, Grants.gov publication of the 
Basic Center Discretionary Grants program announcement (HHS-2004-ACF-ACYF-CY-0063). We 
appreciate the considerable effort that went into the preparation of your application. 

I regret to inform you that your application was deemed non-responsive and was not considered under this 
funding announcement. The limited availability of funds permitted us to select only the highest scoring 
applications that also met all of the eligibility requirements. 

Please note Part III (Eligibility Information) of the program announcement. Section 3, Other 
(Disqualification Factors) which determined your application’s non-responsiveness. 

Applications that exceed the ceiling amount will be deemed non-responsive and will not be 

considered for funding under this announcement; or 

Any application that fails to satisfy the deadline requirements referenced in Section IV. 3 

will be deemed non-responsive and will not be considered for funding under this announcement. 

I also want to take this opportunity to tell you about a new Federal website entitled Grants.gov. Grants.gov 
allows organizations to electronically find and apply for all Federal competitive grants. Grants.gov is THE 
single access point for over 900 grant programs offered by the 26 Federal grant-making agencies. At 
http://Grants.gov you can register to receive electronic notification of all future federal funding opportunities, 
including the FY 2007 Basic Center Program Announcement. 

Sincerely, 

Daphne Weeden, Director 
Division of Discretionary Grants 
Family and Youth Services Bureau 

cc: Contact 
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Appendix I: Administration for Children and 
Families’ Notification Letters to Grant 
Applicants 



September 28, 2008 

«AR Name» 

«AR Title» 

«Organ i zation» 

«Address» 

«Address» 

In reference to grant application «Application Number». 

Dear «AR»: 

Thank you for submitting an application in response to the June 3, 2008, Granls.gov publication of the 
Street Outreach Program Discretionary Grants program announcement (HHS-2008-ACF-AC YF-YO- 
0128). 

A panel of non-federal experts who have broad experience in the subject areas covered by the 
announcement carefully reviewed all of the applications received in response to the Street Outreach 
program announcement. The reviewers evaluated the applications against the published criteria, and their 
assessments were the principal basis for the final funding decision on each application. 

Funding decisions have been made, and I regret to inform you that your application was not selected for 
funding. The amount of funds available permitted us to select only a limited number of applications for 
funding. I hope the enclosed summary of your application's strengths and weaknesses will be of 
assistance to you in preparing future applications. 

Please note that some high ranking applications were not awarded in accordance with Part V.2 (Review 
and Selection Process) of the program announcement, which states, “Highly ranked applications are not 
guaranteed funding because of other factors are taken into consideration. These include, but are not 
limited to geographic distribution, previous program performance of applicants, compliance with grant 
terms under previous HHS grants, audit reports, an applicant's progress in resolving any final audit 
disallowance on previous FYSB or other Federal agency grants. 

We appreciate the effort that went into the preparation of your application and look forward to your 
participation in future competitions. 




Curtis O. Porter 

Acting Associate Commissioner 
Family and Youth Services Bureau 

Enclosure 



Page 28 



GAO-10-335 Runaway and Homeless Youth Grants 







Appendix II: Comments from the Department 
of Health and Human Services’ 
Administration for Children and Families 




APR 3 0 2010 

Kay E. Brown, Director 
Education, Workforce, 
and Income Security Issues 
U.S. Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Dear Ms. Brown: 

Enclosed are the Departments comments on the U.S. Government Accountability Office’s 
(GAO) draft report entitled: “Runaway and Homeless Youth Grants: Improvements Needed in 
the Grant Award Process”(GAO-10-335). 

The Department appreciates the opportunity to comment on this report before its publication. 




Acting Assistant Secretary for Legislation 

Enclosure 
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The Department appreciates the opportunity to comment on this Government 
Accountability Office (GAO) draft report. 

GAO Recommendation 

To enhance transparency and fairness in the grant award process, and improve grantees 
ability to plan for services, we recommend that the Secretary of Health and Human 
Services direct the Assistant Secretary for the Administration for Children and Families 
to take the following seven actions: 

• Review and revise announcements to ensure that all evaluation criteria listed 
in the announcements are clearly labeled as evaluation criteria and contained 
in a single section of the announcement. 

• Select peer reviewers with expertise in the program for which they are 
evaluating grant applications. 

• Make all meetings for peer reviewers, including those for new reviewers and 
chairs, mandatory. 

• Document and maintain records of ACF comments to peer review panels 
during the review process. 

• Document the specific reasons for denying grants to high scoring applicants in 
favor of other applicants for the agency record. 

• Provide clear information to applicants about the specific reasons their 
applications were not approved. 

• Notify applicants about the outcome of their application as soon as grant 
award decisions are made. 

ACF Response to GAO’s Recommendation 

GAO found that the criteria upon which grant applications were evaluated were not 
clearly identified or presented in a single location in the announcement. ACF uses a 
standard template for its Funding Opportunity Announcements (FOAs). Section V of the 
template entitled “Applicant Review Information” is where the criteria for evaluation are 
listed. This section is the only section in the FOA that contains the points for each of the 
criteria to be evaluated. 
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Section V of the Street Outreach Program announcement contained the review and 
scoring information for applications submitted. Each element of Section V contained the 
criterion of the element along with the points assigned to each element. This was the 
only section of the announcement where the application was scored based upon the 
applicant’s response to each criterion. 

With respect to the recommendation regarding peer reviewers, given the relatively small 
universe of runaway and homeless youth programs nationally and that FYSB has 
historically excluded reviewers from organizations applying for funding in the same 
competition, FYSB has elected to accept reviewers who are knowledgeable of the risk 
factors faced by runaway and homeless youth. Often these individuals are persons who 
may work in a Basic Center or Transitional Living Program and are responsible for 
reviewing Street Outreach applications. It is important to realize that while a person may 
not work directly in a street outreach program, if they work in a Transitional Living or 
Basic Center Program, they do have outreach experience as both programs have outreach 
components. 

FYSB has also used retired Runaway and Homeless Youth grantee staff that may at the 
time of the reviews serve as consultants. Furthermore, the Bureau has been committed to 
using college students as reviewers, some of whom may have been runaway and/or 
homeless youth themselves. These youth offer valuable insight into the youth service 
needs of this vulnerable population. 

FYSB understands that many professional disciplines often intersect with runaway and 
homeless youth and have not barred them from participating in the review process. 

These disciplines include educators, social workers, therapists, juvenile probation officers 
and retired law enforcement officers. 

Noting GAO’s concern in this area, FYSB will make a concerted effort to ensure that all 
reviewers possess extensive knowledge in the field of the specific runaway and homeless 
youth program for which they are reviewing. However, where this is not possible due to 
the lack of available reviewers, FYSB will work to ensure that each panel has at least one 
person on it with extensive knowledge and experience in the particular program for 
which they are reviewing. 

All orientation and meeting sessions are mandatory. However, due to challenges in travel 
schedules and other unforeseen factors, it has not always been possible for all reviewers 
and chairpersons to attend the sessions when they are given. 

FYSB has attempted to mitigate potential problems associated with reviewers and 
chairpersons inability to attend the training sessions in a timely manner by hosting 
webinar trainings prior to the on-site reviews. FYSB staff also attempt to meet with any 
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individuals who may have missed a meeting in order to bring them up to speed on the 
information prior to the review panels convening. 

Noting GAO’s concern, FYSB will officially notify all reviewers and chairpersons 
participating in future reviews that all training is mandatory. Furthermore, in the event 
that some reviewers and chairpersons are not able to attend the mandatory training 
sessions due to unforeseen circumstances, FYSB will offer make-up sessions prior to 
them joining their review panels. 

FYSB will work to include greater details concerning geographic distribution in the letter 
to unsuccessful applicants. Furthermore, the letter will direct applicants to the FYSB 
website to in order that they may view the list of grantees funded nationally as well as 
identify existing grantees in their State and local community. 

In accordance with ACF policy and procedures, every unsuccessful applicant is entitled 
to an explanation of the reasons why the application was not funded. No Fund letters 
contain a summary of the strengths and weaknesses of the application. Upon request, 
applicants may request a debriefing by the Program Office. 

ACF’s current practice and policy is to notify applicants as soon as grant award decisions 
have been made. ACF agrees that grantees should be notified about the outcome of their 
applications as soon as grant award decisions are made. Award decisions sometimes 
change as a result of the negotiations. There is a possibility that an applicant could deny 
funding or that ACF would reduce the amount of funding for an award, which would then 
make funds available to add additional awards. As part of the grant application process, 
ACF will explain that final grant decisions depend on the results of the grant award 
negotiations between ACF and the prospective grantee, so the results will not be known 
until the negotiation process is completed. 
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GAO’s Mission 


The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO’s 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 


Obtaining Copies of 
GAO Reports and 
Testimony 


The fastest and easiest way to obtain copies of GAO documents at no cost 
is through GAO’s Web site (www.gao.gov). Each weekday afternoon, GAO 
posts on its Web site newly released reports, testimony, and 
correspondence. To have GAO e-mail you a list of newly posted products, 
go to www.gao.gov and select “E-mail Updates.” 


Order by Phone 


The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black and 
white. Pricing and ordering information is posted on GAO’s Web site, 
http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional information. 


To Report Fraud, 
Waste, and Abuse in 
Federal Programs 


Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 
E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470 
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U.S. Government Accountability Office, 441 G Street NW, Room 7125 
Washington, DC 20548 
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Chuck Young, Managing Director, youngcl@gao.gov, (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548 
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